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BACKGROUND 


Visual  target  detection  is  a  first  step  in  ttirgei  identification  and  the  visual  control  of 
behavior  in  both  natural  and  artificial  visual'rnotor  systems.  This  paper  presents  a  simple 
algorithm  for  machine  visual  target  detection  and  lociUization  based  on  aspects  of  the 
known  functional  neuroanatomy  of  biological  systems.  The  system  can  generate,  from 
the  relative  motion  of  a  target  image  on  the  sensor  surface,  a  series  of  motor  commands 
that  reorient  the  high-resolution  portion  of  the  sensor  surface  to  regions  of  the  target  that 
potentially  contain  the  most  information.  The  reorientations  are  analogous  to  saccadic  eye 
movements  of  biological  visual  systems. 

After  light  detection,  one  of  the  most  primitive  and  phylogenetically  persistent  of  visual 
functions  is  motion  analysis.  Motion  analysis  is  used  by  insects  (Wehner,  1981)  and  their 
amphibian  predators  (Ewert,  19/8)  tor  target  detection  and  navigation.  Similar 
mechanisms  are  most  likely  used  for  similar  puiposes  in  advanced  mammals. 

The  Y  ganglion  cells  of  the  vertebrate  retina  respond  transiently  to  rapid  change  in 
contrast  (Cleland  et  al.,  1971),  and,  therefore,  are  considered  to  be  suitable  motion 
detectors  (Breitmeyer,  1983).  This  activity  is  sent  to  the  superior  colliculus  (SC)  of  the 
brain-stem  tegmentum  (or  to  its  comparative  region,  the  optic  tectum  in  other  venebrates 
[Fite  &  Scalia,  1976]).  The  superior  colliculus  is  organized  retinotopically  (Cynader  & 
Berman,  1972).  Straschill  and  Rieger  (1973)  were  able  to  evoke  eye  movements  through 
electrical  stimulation  of  the  superior  colliculus  (SC).  The  targets  of  the  evoked  eye 
movements  were  regions  of  the  visual  space  that  normally  were  represented  by  afferents 
to  the  area  of  the  superior  colliculus  receiving  the  electrical  stimulation.  After  integration 
and  comparisons  of  input,  the  SC  produces  a  specific  set  of  signa's  to  motor  nuclei.  These 
signals  reflexiveiy  initiate  commands  to  the  eye,  neck,  and  trunk  muscles  (via  additional 
brain  stem  and  spinal  nuclei)  to  orient  the  sensor  or  the  entire  organism  to  the  region  of 
space  where  the  dominant  activity  occurred. 

We,  therefore,  created  a  neural  network  architecture  based  on  anatomical  and 
physiological  evidence  from  the  literature  on  the  mammalian  Y  retina  and  superior 
colliculus.  The  algorithms  determine  competitively  the  region  of  greatest  activity  from 
the  sensor  surface  and  return  coordinates  to  alter  the  fixation  point  to  bring  that  region 
onto  the  high-resolution  center  of  the  sensor  surface. 

MODEL  MECHANISMS  OF  TARGET  DETECTION  AND  LOCALIZATION 

The  computational  model  of  the  Y  retina  used  in  this  report  is  based  on  previous  work 
(Blackburn  et  al.,  1987),  but  is  reduced  in  complexity  to  provide  only  for  motion 
detection.  The  model  is  further  modified  to  provide  for  graded  variation  in  receptive  field 
size  with  eccentricity  (distance  from  the  center  of  the  receptor  array)  and  to  accomplish  a 
polar  to  rectangular  mapping  from  receptor  to  computational  fields  (Blackburn,  1993). 
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The  receptor  matrix  is  m.  N  by  N  rectangular  field  (N  =  256),  indexed  by  .v  and  >. 
compatible  with  the  output  of  a  video  frame  grabber.  The  receptive  fields,  composed  of 
groups  of  neighboring  receptors,  are  mapped  to  a  smaller  n  by  m  rectangular  matrix  ( n  = 
m  ~  64),  indexed  by  i  and  j,  that  becomes  the  ganglion  output  layer.  The  mapping  js  a 
log-polar  transformation  based  on  receptive  field  center  locations  and  sizes  that  are 
computed  once  and  stored  in  a  lookup  table.  In  mammalian  retina,  Y  ganglion  cells  are 
absent  near  the  visual  field  center,  but  appear  with  decreasing  concentration  from  a  few 
degrees  eccentricity  into  the  far  periphery  (Perry  et  al.,  1984). 

MOTION  DE'^'ECTION  IN  THE  RETINAL  MODEL 

In  the  current  model  of  the  Y  retina,  outlined  in  figure  1,  the  horizontal  elements  rapidly 
assume  the  potentials  of  their  overlying  receptors,  and  then  counter  receptor  input  to  the 
underlying  bipolar  elements.  These  horizontal  elements  are  modeled  after  the  small  B 
type  described  by  Sterling  (1983)  that  have  no  gap  junctions,  and  synapse  only  w  ith  rods 
and  rod  bipolars.  Both  on-center  and  off-center  bipolar  elements  are  computed.  The  rapid 
matching  of  receptor  potentials  by  the  horizontals  causes  the  bipolars  to  respond 
primarily  to  change  in  receptor  potential  and  cease  to  respond  to  constant  contrast 
spatially  or  temporally. 


Figure  1.  Model  of  Y  retinu  for  motion  detection  showing  receptors  (/?),  horizontals  {//),  on-cen(er  bipolars  {HI  i. 
off-center  bipolars  (BO),  amaerine  (A),  on-center  ganglion  (RYI),  and  off-center  ganglion  (KYO)  elements.  Onl> 
origins  and  terminals  of  lines  make  connecli(in.s.  Inhibition  is  indicate  b>  a  terminal  dot.  Kxcitatioii  is  indicated 
by  a  terminal  arrow. 
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The  amacrine  element  modulates  bipolar  to  ganglion  transmission  to  provide  a  consistent 
mean  level  of  activity  from  the  Y  retina  irrespective  of  light  intensity  and  contrast. 
Individually,  the  Y  ganglion  elements  are  jiggle  detectors,  responding  to  temporal  change 
in  contrast  within  their  receptive  fields.  The  output  of  the  motion  analysis  subnetwork  of 
the  retinal  model  is  an  analog  potential  from  each  ganglion  element  that  represents,  at  any 
moment  in  time,  the  complexity  of  the  image  moving  in  the  receptive  field  of  that 
element. 

The  input  potential  on  the  horizontal  (Hh)  is  the  new  receptor  potential 


Hb 


A-.y 


^x,\ 


[1] 


where  R  is  the  receptor  potential  .and  .v  imd  y  locate  the  receptor  on  the  sensor  surface. 


In  one  time  step  (r),  the  horizontal  potential  equilibrates  to  the  input  potential. 

=^^.v,y(M)-  [2] 

The  bipolar  elements  then  take  the  difference  between  the  overlying  phonsreceptor  and 
the  netirest  horizontal  element.  In  one  case,  the  bipolar  elements  pass  the  excess  of  the 
horizontal  activity  over  the  receptor  average,  while  in  the  other  case  the  bipohir  elements 
pass  the  excess  of  the  receptor  activity  over  the  horizontal.  Bipolar  elements  do  not  pass 
negative  potentials  (they  are  half-wtive  rectified). 

On-center  bipolar  activity  (B1 )  is  the  greater  of  the  receptor  potentials  over  the 
equilibrated  local  horizontal, 

B 1 y  =  XTi'dX  {0, j  - [,^] 

Off-center  bipolar  activity  {BO)  is  the  greater  of  the  equilibrated  local  horizontal  over  the 
receptor  potentials, 

B0x,y  =  max  {0,  H ^  y  1^1 

The  amacrine  element  (A)  is  driven  toward  the  potential  on  the  most  active  bipolar,  then 
dissipates  slowly. 

A(t)=K\*  mux  [5] 

where  K]  0.9. 
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The  on-center  (RYl)  and  off-center  (RYO)  ganglion  output  elements  are  the  sum  of  their 
respective  bipolars  in  a  local  receptive  field  whose  radius  increases  with  eccentricity 


RYOi  j  -  {K21A)*  Z.h,h  ^^x±h,y±h 


RYlij  =  (K2/A)*  IhM  Bl.x±h,y±h  ,  [6] 

where  fC2  is  half  the  maximum  possible  input  potential  (=128),  and  h  ranges  from  0  to  the 
receptive  field  radius  (Blackburn,  1993).  This  information  is  passed  to  the  input  layer  of 
the  model  superior  colliculus. 


SACCADE  GENERATION  IN  SUPERIOR  COLLICULUS  MODEL 

The  SC  model  has  the  surface  dimensions  corresponding  to  the  coordinates  of  the  Y 
retinal  map  (m  by  n).  The  vertical  organization  of  the  SC  is  given  in  figure  2. 


from  ipsilateral  Y  ganglion  elements 


to  motor  controllers 

Figure  2.  Vertical  organization  of  SC.  Input  from  the  Y  retina  is  teruporally  integrated  by  the  SCJn  elements. 
The  Sej  elements  perform  a  local  spatial  summation  and  the  first  SC_i  element  on  any  cycle  that  exceeds  the 
threshold  element  potential  opens  the  transmission  from  the  .SC_m  element  to  flic  SC  out  element.  The 
potentials  from  the  SC_out  element  then  reset  the  threshold.  Inhibition  is  indicated  by  dots,  excitation  is 
indicated  by  arrows.  Crossing  lines  do  not  interconnect. 
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Because  Rashbass  (1961)  found  that  saccades  were  not  made  to  foveaJ  targets  within  a 
visual  angle  of  0.3  degree  from  the  center  of  the  fovea,  and  bectiu.se  Y  ganglion  cells  are 
generally  absent  in  the  fovea,  we  exclude  the  most  central  pixels  of  the  central  receptor 
surface  (fovea)  from  the  target  competition. 

A  global  threshold  (0),  maintained  within  the  SC,  is  distributed  to  all  output  layer 
elements  and  generally  prevents  an  output  saccade  command  from  being  made. 

•  Eventually,  however,  the  potentials  on  the  intermediate  layer  elements  accumulate  and 
one  reaches  a  value  that  exceeds  the  threshold.  At  that  time,  the  inhibition  from  the 

«  threshold  element  to  a  select  group  of  output  elements  associated  with  the  winning 

intermediate  layer  element  is  blocked  by  the  winning  intermediate  layer  element. 

The  two  bilateral  superior  colliculi  are  proposed  to  have  a  mutually  inhibitory 
relationship.  This  inhibition  is  topographic  on  the  SC  surfaces.  Thus,  one  output  element 
in  the  SC  inhibits  one  input  element  in  the  contralateral  SC  that  represents  a  point  on  the 
retina  exactly  equidistant  on  a  line  running  from  the  point  represented  by  the  first  through 
the  center  of  the  retina.  This  relationship  is  illustrated  in  figure  3.  The  log-polar  mapping 
of  the  Y  retinal  to  the  SC  greatly  simplifies  the  topographic  inhibitory  relationships 
between  the  bilateral  SC.  The  result  of  the  mapping  is  that  after  a  saccade,  locations  in 
the  SC  that  were  active  during  the  last  saccade  command  represent  points  on  the  retinal 
surface  that  are  always  contralateral  to  points  that  subsequently  overlay  the  last  saccade 
target.  This  consequence  of  natural  neural  network  geography  allows  an  efficient  means 
to  prevent  a  remm  saccade  to  the  last  target.  The  two  bilateral  superior  colliculi  simply 
and  mutually  inhibit  homologous  points  on  the  log-polar  projection.  Without  such  an 
inhibition,  the  saccades  could  become  locked  in  a  two-point  limit  cycle  between  the  two 
most  attractive  features  in  the  visual  space.  Without  the  log-polar  projection,  the  routing 
of  inhibitory  fibers  between  the  two  SC  would  be  considerably  more  complex. 

The  input  to  the  SC  from  the  Y  retinal  ganglion  elements  is  retinotopically  distributed 
and  integrated  over  time,  allowing  excitation  to  build  up  in  a  local  area  if  there  is  no  or 
relatively  little  inhibition  from  the  contralateral  SC, 

^^^m,n(t) '  ^^_^^^^m,o(t-l),  [7] 

•  where  o  indexes  the  homologous  point  in  the  contralateral  SC  K/s,  is  a 

constant  <  1.0  that  determines  the  decay  of  the  SC  input 
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Figure  3.  Mutual  iohibition  bet^vecn  homologous  points  of  the  superior  colliculus  prevents  return  saccades  to  the 
last  target  When  a  saccadc  is  made  to  target  B  in  (a),  the  last  target  A 1  becomes  relocated  at  A2  on  the  receptor 
surface.  Because  locations  A2  and  B  are  equidistant  from  the  center  of  the  receptor  surface,  and  lie  upon  the 
same  diagonal,  they  also  occupy  homologous  points  on  the  bilateral  log-polar  map  to  the  superior  colliculus  (b). 


The  input  activity  diffuses  locally  to  the  six  nearest  neighbors  in  the  intermediate  layer 


where  i^5  is  a  constant  of  diffusion  <  1.0  and  z  is  the  radius  of  the  local  neighborhood. 

The  locally  spatially  and  temporally  summed  SC  input  potentials  {SC  i/fj  „)  are  compared 

to  the  global  threshold.  The  order  of  the  comparison  is  pert'ormed  randomly  bci vveen  the 
two  bilateral  SC  to  remove  any  bias  in  sampling  the  intermediate  layer  as  a  result  of  the 
serial  computation.  The  intermediate  layer  element  with  the  Icirgest  potential  that  exceeds 
threshold  is  able  to  pass  the  SCjn  activity  out  of  the  SC. 
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[y] 

where  1  and  J  are  the  coordinates  of  the  center  of  a  small  group  of  winning  elements  that 
lie  within  a  radius  z  of  the  center,  and  0  is  the  threshold. 

The  threshold  is  then  reset  to  the  sum  of  the  winning  output  elements 


©  =  YSC_ouii^j+, 


[10] 


All  other  output  elements  remain  at  zero,  and  all  input  elements  are  reset  to  zero. 
The  threshold  slowly  decays  on  each  program  cycle, 

0(r)  =  -^3*Q(M)’  [11] 


where  Kj  is  a  constant  <  1 .0, 

The  new  target  coordinates  are  determined  by  the  location  of  the  winning  elements  in  the 
input  field  and  their  levels  of  activity.  The  final  output  is  the  vector  sura  of  these 
contributions  as  has  been  described  for  the  biological  superior  colliculus  (Lee  et  al. 
1988).  This  computation  is  presumably  carried  out  in  biological  systems  in  retinal 
rectangular  coordinates  in  the  motor  nuclei  that  receive  SC  output.  In  the  present 
implementation,  the  changes  in  fixation  in  terras  of  retinal  and  y  cvK)rdinates  required  to 
foveate  the  new  target  is  given  by 

A>’  =  (K^/SCji  j)*1.  (L/;^j+2-cen/cr)*5C_oMr/j^j+j  ,  [12] 


where and  yj±zJ±z^^  retiaal  map  coordinates  of  the  winning  SC  elements 
recovered  from  the  lookup  table,  and  is  as  defined  for  equation  [9]. 

In  the  present  implementation.  Ax  and  Ay  are  sent  to  two  virtual  motors  that  move  the 
point  of  fixation  on  the  X  and  Y  axes  through  the  xTsual  field.  The  SC_out  potentials  of 
the  neighborhood  that  wins  the  competition  to  control  the  saccade  are  multiplied  by  the 
difference  between  their  transformed  coordinates  and  the  center  of  the  receptor  field. 
These  differences  can  be  considered  as  fixed  weights  for  the  A’  and  Y  directions  that  tnust 
be  traversed  to  target  the  region  responsible  for  the  winning  activation.  Tho.se  sums  of 
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prcxlucts  are  adjusted  by  the  neighborhood  sum  {5r  /)  to  achieve  the  weighted  vector 
average  for  movement  on  A’  or 

The  present  sysm:r  has  a  very  simple  efiecior  capability,  limited  to  selecting  a  different 
portion  of  the  cideo  frame  to  pass  to  the  retina.  Theiefore,  movement  is  cuiTcntly 
performed  .electronically  within  the  limiLs  of  the  video  frame.  The  video  camera  provides 
a  512  by  512  pixel  image  encompassing  about  22  degrees  of  visual  angle.  A  25b  by  256 
sample  of  the  image  is  passed  to  the  retina.  ' 

In  the  process  described  so  far,  the  Y  retina  detects  imttge  motion  of  a  moving  target.  T  he  * 

Y  retina  then  sends  this  information  to  the  SC  where  the  most  active  region  of  the  visual 

field  is  determined,  and  coordinates  for  motor  commands  to  bring  the  .,eriier  of  the 

receptor  surface  upon  that  region  of  visual  space  are  computed.  This  system  wiil. 

however,  detect  potential  targets  only  if  the  targets  are  moving.  Static  Uirgets  will  be 

ignored.  A  .simple  .solution  to  delect  .static  targets  has  been  developed  for  higher 

vertebrates  that  can  easily  move  their  eyes,  tye  movement  creates  a  i  .dative  motion  of 

static  objects  on  the  retina.  This  allow's  the  detection  and  localization  of  immobile 

animate  objects  as  well  as  inanimate  objects.  The  comeiaiion  between  the  ability  to  move 

the  eyes  and  the  presence  of  a  high  resolution  fovea,  reported  by  Ali  and  Klyoe  ( !  985),  is 

not  unexpected. 

In  the  present  model,  when  the  threshold  decreases  to  below-  50%  of  its  post  saccade  reset 
value,  small  oscillations  in  fixation  occur.  These  oscillations  are  sufficient  to  cause  the 
motion  analysis  subnetwork  of  the  retina  to  detect  the  relative  motion  of  regions  of 
contrast  on  the  receptor  surface.  Since  this  output  is  proportional  to  the  distribution  of 
contrast  among  the  receptive  fields  of  the  Y  ganglion  elements,  a  competition  for  high- 
contrast  regions  in  the  peripheral  visual  field  is  possible.  In  the  event  that  the  immediate 
visual  field  contains  no  detectable  contrast,  the  intenial  noise  of  the  SC  will  eventually 
generate  a  saccade  command  to  a  random  location  as  the  threshold  further  drops. 

BEHAVIOR  OF  THE  MODEL 

Scan  paths  to  moving  patterns  are  given  in  figure  4  a-c.  The  results  reported  here  were 
obtained  in  numerical  simulations  on  a  Convex  C-24d  minisupercomputer,  \  ideo  camera 
input  generates  similar  behavior.  The  sizes  of  the  patterns  were  slowly  increased  and 
decrea.sed  about  a  common  point,  simulating  motion  toward  or  away  from  the  observing 
receptor  surface.  The  overlaid  pairs  of  patterns  in  the  figures  reflect  the  size  limits  w  ithin 
which  the  patterns  were  varied.  The  change  in  size  was  about  15%  on  any  dimension.  The 
target.s  of  each  .saccade  are  indicated  by  the  arrowhead  locations,  and  the  order  of 
saccades  are  indicated  by  the  connecting  lines.  .Most  saccade  targets  were  located  on  the 
perimeters  of  the  patterns.  This  occuned  because  the  targeting  mechanism  was  sensitive 
only  to  contrast,  and  because  the  peripheral  receptor  surface  was  more  sensitive  to 
contrast  than  was  the  central  receptor  surface  due  to  the  larger  peripheral  receptive  fields 
over  which  potentials  were  integrated.  Longer  saccades  were  thus  favored  over  shorter 
saccades  when  a  potential  target  was  present.  While  st)me  regions  of  the  patterns  attracted 
more  saccades  than  others,  target  locations  varied  considerably. 
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Figure  4.  Scan  paths  to  moving  patterns  (a-c). 
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Scan  paths  for  three  static  patterns  are  shown  in  figure  5  u-c.  Regions  of  the  image  field 
were  detected  by  the  spontaneous  jitter  of  the  receptor  surface  that  created  a  relative 
motion  of  the  image.  The  figures  show  that  within  a  sequence  of  less  than  12  saccades. 
most  of  the  high-information  regions  of  the  patterns  were  placed  on  the  high-resoluticm 
portion  of  the  receptor  surface. 

The  scan  paths  for  static  patterns  discovered  by  automotion  (jitter)  are  very  similar  to  the 
scan  paths  evoked  by  moving  stimuli  (figure  4).  The  auiomotion  generated  saccades  did 
visit  the  centers  of  the  patterns  more  often  than  the  evoked  saccades.  but  this  is  likely  an 
artifact  of  the  algorithm  that  changed  the  size  of  the  images.  In  the  dynamic  condition, 
lines  located  at  short  disutnces  from  the  expansion  point  changed  location  less  often  than 
lines  located  at  long  distances  because  of  the  floating  point  to  integer  conversions 
required  in  the  algorithm,  whereas  all  lines  were  moved  in  the  jitter  generation  algorithm 
during  each  oppormniiy. 

SCALE  INVARIANCE 

One  possible  way  to  achieve  scaled  output  is  through  topographical  mapping.  W'hen  the 
motor  map  is  isomorphic  with  the  sensory  map,  an  activation  in  the  sensory  plane  vsill 
automatically  be  scaled  to  the  motor  response  appropriate  to  the  location  of  the  stimulus. 
Thus,  a  large  input  pattern  will  produce  a  large  response  and  a  small  input  will  evoke  a 
small  response,  figure  6  shows  the  scan  path  to  an  image  that  is  tw'ice  the  size  of  the 
image  in  figure  5b.  While  the  sequence  of  targets  differed  between  the  scan  paths  of 
figure  5b  and  figure  6,  many  of  the  same  targets  are  visited.  Importantly,  for  the  issue  of 
scale  invariance,  the  s’ifference  in  image  size  did  not  change  the  relative  importtince  of 
target  regions. 

The  SC  in  the  present  model  is  intended  to  perform,  functions  of  the  mammalian  SC.  that 
is,  to  orient  the  subject  to  potentially  significant  visual  stimuli.  To  do  this,  the  SC  must 
integrate  relevant  information  and  organize  a  response  that  would  properly  reposition  the 
point  of  vi.sual  fixation.  The  requirement  is  to  align  the  center  of  the  receptor  surface  v\ith 
the  center  of  mass  of  the  most  attractive  region  of  the  visual  field.  The  most  attractive 
region  is  defined  in  the  present  algorithm  as  that  region  of  the  visual  field  that  contains 
the  greatest  contrast.  The  detected  level  of  contrast  is  a  function  both  of  the  complexity  of 
the  local  pattern  of  bright  and  dark  regions  and  of  its  motion.  The  horizontal  layer  of  the 
Y  retina  washes  out  any  contrast  that  does  not  move  on  the  receptor  surface. 
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(a) 


(b) 


Figure  5.  Scan  paths  to  static  patterns  (a-c). 

To  demonstrate  the  attractor  mechanism  of  saccade  targeting,  the  pattern  of  figure  6  was 
modified  to  increase  the  contrast  of  the  eyes  and  mouth  by  a  factor  of  three  selative  to  the 
remainder  of  the  pattern  (which  was  twice  as  bright  as  the  background).  The  resulting 
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Figure  6.  Scan  path  to  a  static  image  that  >ras  twice  the  sir.e  of  those  represented  in  figures  4  and  5. 


scan  path  to  the  static  pattern  (figure  7)  clearly  shows  a  limit  cycle  involving  the  three 
targets.  Return  saccades  to  the  mouth,  which  represents  a  more  attractive  target  due  to  its 
size,  are  prevented  by  the  mutual  inhibition  between  homologous  locations  in  the  two 
bilateral  SC  until  after  another  target  has  been  selected. 


Figure  7.  Scan  path  to  static  pattern  when  selected  features  arc  enhanced.  The  brightness  of  eyes  and  mouth  are 
three  times  the  brightness  of  the  remainder  of  the  pattern  . 
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